Pairing Instability in a Nematic Fermi Liquid 
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Nematic Fermi liquid arises when the system of interacting fermions spontaneously breaks the 
rotational symmetry while the translational symmetry is preserved. We consider a Nematic Fermi 
liquid of fermions with two distinct quantum numbers. Two quantum numbers represent the spin 
degrees of freedom or the layer degrees of freedom of the bilayer system. There are two director 
modes in this system: the in-phase and out-of-phase modes. We find that the out-of-phase (in- 
phase) mode mediates an attractive (repulsive) interaction between fermions with different quantum 
numbers. Pairing instability occurs under certain conditions when the in-phase mode is gapped due 
to a lattice potential. Implications to the stripe phase of Cuprates and the quantum Hall effect of 
two dimensional electron systems are discussed. 

PACS numbers: 74.20.-z, 74.20.Mn, 71.10.Hf 
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a. Introduction The occurrence of the novel 
anisotropic metallic states in strongly correlated electron 
systems has been demonstrated by the discoveries of 
the stripe phases in Cuprates [jj] ^ and the anisotropic 
metallic states in high Landau levels of two dimensional 
electron systems in the quantum Hall regime || ||, |J. 
The anisotropic states in these systems are close to the 
unidirectional charge-density-wave states and arise due 
to strong electron-electron interactions. In analogy to 
classical liquid crystals, nematic Fermi liquid state has 
been proposed as a primary candidate for the expla- 
nation of the quantum liquid states mentioned above 
P> 0> Rj R] • I 11 nematic Fermi liquids, the rotational sym- 
metry is spontaneously broken while the translational 
symmetry is still preserved. There are two approaches to 
get nematic Fermi liquid states. One may consider first a 
smectic Fermi liquid state where both the rotational and 
translational symmetries are spontaneously broken |6|, f7| . 
The melting of the smectic state via the proliferation of 
appropriate topological defects will lead to a nematic 
Fermi liquid state. Another approach is to introduce an 
interaction in ordinary Fermi liquid state such that the 
rotational symmetry will be spontaneously broken upon 
the tunning of the interaction. 

Due to the broken translational symmetry, the smec- 
tic state is often visualized as a liquid version of a uni- 
directional crystalline state || ^[ ||, [|. The low energy 
excitations come from the one-dimensional electronic de- 
grees of freedom that reside along the stripes in Cuprates 
or along the edges of two different quantum Hall states in 
high Landau levels of two dimensional electron systems. 
The existence of the fermionic degrees of freedom adds a 
major difficulty in describing the melting of the smectic 
state to a nematic Fermi liquid state. So far, no well- 
defined procedure has been discovered in this direction. 
On the other hand, the second approach has be en p roved 
to be useful. Oganesyan, Kivelson and Fradkin |M intro- 
duced a two-body interaction between quadrupolar den- 
sities of spinless fermions. Upon the tunning of the inter- 
action, the rotational symmetry is spontaneously broken. 
The Goldstone mode of the nematic state is the director 



mode that corresponds to the angular rotation of the 
principal axis of the distorted Fermi surface. Its disper- 
sion relation is highly anisotropic in space in contrast 
to the case of the broken spin symmetries. The interac- 
tion between the fermions and the anisotropic Goldstone 
mode leads to anisotropic self-energy of the quasiparticles 
that show non-Landau-Fermi liquid properties in certain 
directions. That is, the life time of the quasiparticles 
strongly depend on the spatial directions. 

In this paper, we follow the second approach and in- 
vestigate a pairing instability of a nematic Fermi liquid 
of two species of fermions. Two species represent either 
the spin degrees of freedom or the layer degrees of free- 
dom of the bilayer system. Introducing an additional 
interaction between quadrupolar densities of two degrees 
of freedom, we find a nematic Fermi liquid state where 
there are two director modes that corresponds to the in- 
phase and out-of-phase modes of the angular rotations 
of two Fermi surfaces. In general, the out-of-phase mode 
is gapped while the in-phase mode remains gapless. It 
is found that the Goldstone mode of the nematic Fermi 
liquid state of the spinless fermions always mediates a re- 
pulsive interaction. On the other hand, in the case of the 
two species of fermions, the out-of-phase mode mediates 
an attractive interaction between fermions that belong to 
different species while the in-phase mode leads to a repul- 
sive interaction. Since the out-of-phase mode is gapped, 
one expects that the in-phase mode will dominate in the 
low energy limit and the interaction between the fermions 
will be repulsive. In the presence of an external potential 
like a lattice potential, however, the in-phase mode will 
be also gapped. We find in this case that the combined 
total interaction due to the in-phase and out-of-phase 
modes can be attractive so that there will be a pairing 
instability between the fermions of different species. 

b. Choice of the Hamiltonian We consider the fol- 
lowing Hamiltonian for a two dimensional fermion system 
with two quantum numbers, f and j. 
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+ f d 2 rd 2 r'^ 2 (r-r')^Tr[Q Q (r)Q Q (r')] 
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where =|, |, is an annihilation operator of the 
fcrmion with "spin" a, and e(— zV) is the kinetic energy 
operator. Here the matrix quadrupolar density of the 
fermions is defined as 
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The interactions, -F 2 (q) between the fermions with the 
same "spin" and G 2 (q) between the fermions with differ- 
ent "spin" , are given by 



^ 2 (q) 
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(3) 



The details of the form of the interaction do not matter 
as far as the short range interaction is concerned. 

Using a Hubbard-Stratonovich transformation, we can 
write the effective Lagrangian as 

C = /d 2 r^^( r , r )(a T + e (- i V)- M )* ct (r,r) 

J a 

l! ' q F 2 (q)[l+ 3 (q)] Tr[Q + ( q ,r)7V + (- q ,r)] 
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where Q± (q, r) = Q T (q, r) ± Q T (q, r) with 
Qa(q,r)=^< k _ q (r 
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value of N^ p 2 (A^L P 2 ) determines the amount of the sym- 
metric (antisymmetric) rotations of the principal axis of 
the t and J, Fermi surfaces. The saddle point solution de- 
pends on the coefficient of the quadratic term in the long 
wavelength (q — > 0) expansion of the Landau-Ginzburg 
free energy. 



T(N+, AA) = M+Tr[A 2 ] + M_Tr[A 2 ] + • 



(7) 



where • • • represents the quartic and higher order terms. 
One can show that 



M+ - 2(1 +g)F 2 + (I + g) 2 N(0)F 2 



M_ 



2(1 - g)F 2 + (1 - g) 2 N(0)F 2 , 
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where g = g(q = 0) and A(0) is the density of states at 
the Fermi level. We will choose F 2 > and G 2 < such 
that M_|_ < 0. One can see that M_ > in this case. 
Then the saddle point solution, ^ and = 0, 
represents the symmetric distortions and rotations of the 
t and I Fermi surfaces. 

The Gaussian fluctuations of N± (q) = + SN± (q) 
about the nematic liquid ground state and their interac- 
tion with the fermions can be written as (a = 1,2) 
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where C sp describes the mean field nematic liquid ground 
state. 

The propagators of the Gaussian fluctuations, D± ab 
can be obtained at the Gaussian level (Random Phase 
Approximation) as follows. 



*a,k(T)(5) £±.U(q, *"») - A±, ab + (1 ± .g) 2 F 2 n ah (q, »!/„), 
with 



and 6* k is the angle between k and the x-axis. Here 
g(q) = G 2 (q)/F 2 (q) and A±(q, r) are the Hubbard- 
Stratonovich matrix fields given by 
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A + (q t)= ( N ±^> t ) N ±M<1,t) 
±lQ ' j ^±, 2 (q,r) -JV±,i(q,T) 



(6) 



and 



n a6 (q,z^) = A(0) 



Notice that, at the saddle point of the action, N±(q) 

corresponds to (Q±(q)). 

c. Saddle Point and Gaussian Fluctuations Now 
one can integrate out the fermionic degrees of freedom 
and get an effective action in terms of N + (q) and A_ (q) . 
In the putative nematic liquid state, there should be a 
uniform saddle point solution N± ^ 0. Notice that non- 
zero N^ p 1 (iVf p 1 ) will lead to the symmetric (antisymmet- 
ric) distortions of the f and J, Fermi surfaces. The finite 
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where </> is the angle between q and the x-axis. Here 
v n = 2nirT is the Matsubara frequency and s = iu n /vpq- 
A± ; i and A± j2 represent the gap in each collective mode, 
that depend on the interaction strength, but their pre- 
cise forms are not important in the following analysis. In 
the nematic liquid state, for generic interactions without 



an external perturbation, we expect that there should be 
one gapless Goldstone mode, the in-phase director mode, 
associated with 8N +y2 . That is, A +i 2 = 0. If the system 
experiences a weak square lattice potential or a weak 
quadrupolar external perturbation, even the in-phase di- 
rector mode will be gapped. As will be shown later, both 
the in-phase (8N +t2 ) and out-of-phase (8N_ i2 ) director 
modes will be important in the analysis of the pairing 
instability. The distortion modes, 8N+1 and 8N- t \, are 
gapped even in the absence of the interaction between f 
and I fermions, thus one expects that the gap of these 
modes will be even bigger. We will drop the distortion 



modes in the analysis from now on. 

The propagators of two director modes in the long 
wavelength and low energy limits can be obtained as 

D ± \ 2 = A± 2 + K± q 2 + 2(1 ± g) 2 AT(o)F|^sin 2 (20) 

where, in the absence of the external potential, A +! 2 = 
and A_ 2 > 0. When there is a weak quadrupolar exter- 
nal potential, we expect A_ 2 > A + 2 7^ 0. Here k + and 
k_ are the stiffness coefficients and their detailed depen- 
dence on the interaction strength will not be important. 



d. Effective Two-Body Interaction The fermions with f and j. quantum numbers interact each other by exchanging 
two director modes just like the case of the phonon mediated electron-electron interaction. The effective two-body 
interaction between fermions can be obtained by integrating out the director modes in the effective action given by 
Eq.^)| and using the following form of the coupling between the director modes and the fermions, 

(I + g)5N +a {c i ) sin ( 2 ^)< k _ q *a,k + (1 - 9)8N., 2 (q) sm(24)< u _ q <3*/5, k . (14) 

k k 

After integrating out the director modes, the effective interaction between fermions is obtained as 

S 'eS = EE E E Fl ( k ' k ^' W ™)*lk- q ( W m-^)*Lk'+ q (< + ^)*c t ,k'(^m)*a,k(Wm) 
q,!Ai k,w m k',u' m a 

+ E^( k ' k ''^^")*-k- q ( W ™-^)*kk'+q(< + ^0*/3,k'(<)*a,k(^m), (15) 

q,t>„ k,(^ m k' ,u)' m 

where 

^(k,k',q,iiv) = F 2 2 [(l +g) 2 D + . 22 {<i,iv n ) + (1 - 3 ) 2 Z?_, 22 (q, iv n )\ sin(2fl k ) sin(20 k O 
V r 2 (k,k',q,iv n ) = F 2 [{l+g) 2 D + , 22 (qJis n )-(l-g) 2 D^, 22 ((i,iu n )}sm(28 k )sm(28v). (16) 



e. Cooper Channel and Pairing Instability Notice 
that V\ and V 2 correspond to the interactions between 
the fermions with the same spin and with different spins 
respectively. In the Cooper channel, k' = — k and 
$k' = $k + 7T) V\ is always repulsive while V 2 can be 
attractive if , g\ = \G 2 \/F 2 > 1. In the long wavelength 
and low energy limit, V 2 in the Cooper channel can be 
written as (here A + = A + .2 and A_ = A_, 2 ) 



y 2 (k, -k, q,iv n ) 



Asin 2 (26» k ) 



C + B^sm 2 (2q>y 



(17) 



where 



A = Fl [ 2\g\(A + + A_) - (\g\ 2 + 1)(A_ - A+) ] 

B = 2N(Q)F 2 2 [(| 5 | 2 + I)(A + + A_)-2| 3 |(A_-A + )] 

C = A + A_. (18) 



One can show that V% < if the following condition is 
satisfied. 



A_ - A+ - 131 - A_ - A+ (19) 
/. Gap Equation and Nematic Superconductor Us- 
ing the attractive interaction V 2l the gap equation for 
A(k) = (^^k^i.-k) can be written as 



V 2 (k, -k, k - k', iv n - W ;)A(k', iv> n ) 
K) 2 +e+A 2 (k',z<) 



A(k>„) = -TE 

(20) 

Assuming that A(k, iv n ) depends mostly on the angle # k , 
we can perform the integral over £ k / = e k > — /i and get 
the following results at T=0. 
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where 9 = 9^—9^. We also use q w hp \6\ and 
sin 2 (20) ss sin 2 (20k) in the long wavelength and low fre- 
quency limit. One can easily see that A(#k, iv n ) vanishes 
at 6*k = 0, 7r. It can be shown that the frequency de- 
pendence in A(#k, iv n ) is weak near 6>k — 0, ±§,7r. When 
A(6» k ) sin 2 (26> k ) < E e &, we get 



A(#k) 



where 



E, 



sin 2 (26» k ) 



exp 



A(O)T4ffsin"(20 k ) 



26» c C , 

— wf«_f 

e B 

9cA 

7T C 



(22) 



(23) 



Here 9 C is the cut-off in the integral over the angle 9 in 
Eq.pl|. This is the asymptotically correct solution near 
#k = 0, ±5, 7r. Thus the gap is exponentially suppressed 
in the four directions. We have to mention, however, that 
the whole angular dependence of the gap will depend on 
the underlying lattice structure. 

g. Summary and Conclusion We study the pairing 
instability in nematic Fermi liquid states where the orien- 
tational order is spontaneously broken due to an electron- 
electron interaction. In the case of the fermions with no 
internal quantum number (e.g. spin polarized electrons), 
the director mode - the goldstone mode of the nematic 
order - always mediates a repulsive interaction in the 
Cooper channel. On the other hand, when the fermions 
carry two different quantum numbers (e.g. spin or the 
layer index of the bilayer system), two independent di- 
rector modes exist. The out-of-phase (in-phase) mode 
mediates an attractive (a repulsive) interaction in the 
Cooper channel. When an external potential is applied 
to the system so that the in-phase mode becomes gapped, 
the total effective interaction can be attractive under cer- 
tain conditions. We showed that this leads to a pairing 
instability. The resulting pairing gap has a remarkable 



structure. There are four nodes where the gap vanishes 
and the gap is exponentially suppressed near the nodes. 
The angular dependence near the nodes cannot be de- 
scribed by an analytic expansion in the angle around the 
nodes. 

Our results may have the following implications to the 
physics of Cuprates and quantum Hall effect. 1) The oc- 
currence of the superconductivity in the stripe phase of 
Cuprates has been a subject of great interest. In particu- 
lar, the question of whether the stripes (or some dynamic 
generalization) are in favor of or against superconductiv- 
ity has not been clearly answered [0. Our work on the 
pairing instability of a nematic Fermi liquid state will 
shed some light on this question especially because the 
existence of the nematic order and its (pseudo-) Goldstone 
mode is crucial for the effective attractive interaction and 
pairing instability. 2) In the lowest Landau level, the 
Fermi- liquid-like state |11 of the filling factor v = 1/2 
can be described as a Fermi liquid state of the quasipar- 
ticles called composite fermions jl^]. There have been a 
number of examples of the paired quantum Hall states 
in the bilayer generalization, where the ground state can 
be described as a paired state of the composite fermions 
|l{|. In the half- filled high Landau levels, the 
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ground state is a highly anisotropic metallic state very 
close to a unidirectional charge density wave If one 
can consider the nematic Fermi liquid state of the com- 
posite fermions as a candidate for the anisotropic state 
mentioned above p0| , one would expect that an unusual 
paired quantum Hall state may occur in high Landau 
levels in the bilayer system, arising from the pairing in- 
stability of the underlying nematic Fermi liquid states of 
the composite fermions. 
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